Gated Attention

Researchers introduced Tapered Language Models (TLMs), an architectural principle that allocates more parameter capacity to earlier layers and less to later layers under a fixed budget. Across four ar…

Alibaba's Qwen3.6-27B dense model outperformed its own 397-billion-parameter MoE predecessor on SWE-bench Verified (77.2% vs. ~76.2%) while running on a single RTX 4090, challenging the prevailing MoE…